System and method of controlling construction machinery

ABSTRACT

A control system for construction machinery includes an upper camera installed in a driver cabin of a rear vehicle body to photograph the front of the driver cabin, a lower camera installed in a front vehicle body rotatably connected to the rear vehicle body to photograph the front of the front vehicle body, an angle information detection portion configured to detect information on a refraction angle of the front vehicle body with respect to the rear vehicle body, an image processing device configured to synthesize first and second images captured from the upper and lower cameras, and configured to determine a position of a transparency processing area in the synthesized image according to the refraction angle information and transparency-process at least one of the first and second images in the transparency processing area, and a display device configured to display the synthesized image transparency-processed by the image processing device.

PRIORITY STATEMENT

This application claims priority under 35 U.S.C. § 119 to Korean PatentApplication No. 10-2021-0099121 and Korean Patent Application No.10-2022-0083368, respectively filed on Jul. 28, 2021 and Jul. 6, 2022 inthe Korean Intellectual Property Office (KIPO), the contents of whichare herein incorporated by reference in their entirety.

BACKGROUND 1. Field

Example embodiments relate to a control system and method forconstruction machinery. More particularly, example embodiments relate toa control system for recognizing forward obstacles when working ordriving construction machinery such as a wheel loader, an excavator,etc., and a method of controlling construction machinery using the same.

2. Description of the Related Art

In general, construction machinery such as a wheel loader, an excavator,etc., is widely used to excavate sand, gravel, and the like and load itinto a dump truck. These works may be performed by driving a workapparatus installed in the construction machinery such as a bucket and aboom. However, the work apparatus may obstruct or limit an operator'sfront view while driving or traveling, and thus, obstruction of theoperator's front view by the work apparatus may cause a safety accident.

SUMMARY

Example embodiments provide a control system for construction machinerycapable of improving forward visibility limited by a work apparatus.

Example embodiments provide a control method for construction machineryusing the control system.

According to example embodiments, a control system for constructionmachinery includes an upper camera installed in a driver cabin of a rearvehicle body to photograph the front of the driver cabin, a lower camerainstalled in a front vehicle body rotatably connected to the rearvehicle body to photograph the front of the front vehicle body, an angleinformation detection portion configured to detect information on arefraction angle of the front vehicle body with respect to the rearvehicle body, an image processing device configured to synthesize firstand second images captured from the upper camera and the lower camerainto one image, and configured to determine a position of a transparencyprocessing area in the synthesized image according to the refractionangle information and transparency-process at least one of the first andsecond images in the transparency processing area, and a display deviceconfigured to display the synthesized image transparency-processed bythe image processing device.

In example embodiments, the image processing device may include asteering angle calculator configured to determine a steering hinge angleof the front vehicle body from the refraction angle information obtainedby the angle information detection portion, and a transparency processorconfigured to determine the position of the transparency process area inthe synthesized image according to the determined steering hinge angle.

In example embodiments, the steering angle calculator may convert adetection value obtained from the angle information detection portioninto the steering hinge angle value of the front vehicle body.

In example embodiments, the angle information detection portion mayinclude a center pin angle sensor, a steering cylinder displacementsensor or a gyro sensor.

In example embodiments, the control system may further include a workapparatus posture detection portion configured to detect a posture of afront work apparatus, and the image processing device maytransparency-process the at least one of the first and second images inthe synthesized image according to the posture of the front workapparatus detected by the work apparatus posture detection portion.

In example embodiments, the image processing device maytransparency-process the first image in the synthesized image when atleast a portion of the front work apparatus invades a predeterminedposition, and the image processing device transparency-processes thesecond image in the synthesized image when the work apparatus does notinvade the predetermined position

In example embodiments, the control system may further include an inputportion configured to set an image processing condition in the imageprocessing device.

In example embodiments, the image processing condition may include atransparency processing switching timing of the first and second imagesor the transparency processing area of the entire display area of thedisplay device.

In example embodiments, the image processing device may process that, inthe transparency processing area, an outline of an exterior of a boom ora bucket of the transparency-processed first and second images isdisplayed with a line or a dotted line.

In example embodiments, the outline of the transparency-processed boomor bucket may be displayed by transparency-processing an image of theboom or bucket that is taken from the first image or the second image.

In example embodiments, the image processing device may selectivelytransparency-processes an image of a boom or a bucket coupled to thefront vehicle body in the first and second images in the transparentprocessing area.

In example embodiments, the synthesized image may include an objectrecognized by the image processing device in the first image and thesecond image.

In example embodiments, the image processing device may recognize aperson, animal, building or equipment as an object through apredetermined algorithm.

In example embodiments, the image processing device may extract a cropimage from any one of the first and second images, maytransparency-process a partial region in the other one of the first andsecond images, and may place the extracted crop image in thetransparency-processed region to create the synthesized image.

In example embodiments, the transparency-processed region may correspondto the transparency processing area.

In example embodiments, the image processing device may set a portion ofthe first image as a first synthesis region, may set a portion of thesecond image as a second synthesis region, may translucently process thesecond synthesis region of the second image, and may synthesizes thetranslucently processed second synthesis region with the first synthesisregion of the first image to create the synthesized image.

According to example embodiments, in a method of controllingconstruction machinery, a first image of the front of a driver cabin isobtained from an upper camera installed in the drive cabin of a rearvehicle body. A second image of the front of a front vehicle body isobtained from a lower camera installed in the front vehicle bodyrotatably connected to the rear vehicle body. Information on arefraction angle of the front vehicle body with respect to the rearvehicle body is obtained. The first and second images are synthesizedinto one image. A position of a transparency processing area in thesynthesized image is determined according to the obtained refractionangle information. At least one of the first and second images in thetransparency processing area is transparency-processed. Thetransparency-processed image is displayed through a display device.

In example embodiments, determining the position of the transparencyprocessing area according to the obtained refraction angle informationmay include determining a steering hinge angle of the front vehicle bodyfrom the refraction angle information, and determining the position ofthe transparency process area in the synthesized image according to thedetermined steering hinge angle.

In example embodiments, determining the steering hinge angle of thefront vehicle body from the refraction angle information may includeconverting a refraction angle detection value into the steering hingeangle value of the front vehicle body.

In example embodiments, obtaining the refraction angle information ofthe front vehicle body with respect to the rear vehicle body may includeusing a refraction angle detection sensor, and the refraction angledetection sensor includes a center pin angle sensor, a steering cylinderdisplacement sensor or a gyro sensor.

In example embodiments, the method may further include detecting aposture of a front work apparatus, and transparency-processing the atleast one of the first and second images in the transparency processingarea may include transparency-processing the at least one of the firstand second images in the synthesized image according to the detectedposture of the front work apparatus.

In example embodiments, the method may further include setting an imageprocessing condition under which the at least one of the first andsecond images is transparency processed.

In example embodiments, the image processing condition may include atransparency processing switching timing of the first and second imagesor the transparency processing area of the entire display area of thedisplay device.

According to example embodiments, in a method of controllingconstruction machinery, a first image of the front of a driver cabin isobtained from an upper camera installed in the drive cabin of a rearvehicle body. A second image of the front of a front vehicle body isobtained from a lower camera installed in the front vehicle bodyrotatably connected to the rear vehicle body. Information on arefraction angle of the front vehicle body with respect to the rearvehicle body is obtained. A position of a transparency processing areain the first image is determined according to the obtained refractionangle information. A crop image is extracted by setting a partial regionin the second image corresponding to the transparency processing area asa crop region. Transparency of the transparency processing area of thefirst image is adjusted. The crop image extracted from the second imageis synthesized to the transparency processing area of the first image.The synthesized image is displayed through a display device.

In example embodiments, synthesizing the extracted cropped image to thetransparency processing area of the first image may include adjusting asize of the crop image extracted from the second image, and synthesizingthe adjusted crop image to the transparency processing area of the firstimage.

In example embodiments, adjusting the transparency of the transparencyprocessing area of the first image may include processing such that anoutline of an exterior of a boom or a bucket is displayed with a line ora dotted line in the transparency processing area of the first image.

In example embodiments, the method may further include recognizing aperson, animal, building or equipment as an object through apredetermined algorithm in the synthesized image.

According to example embodiments, a control device for constructionmachinery may synthesize a first image and a second image captured froman upper camera installed in a driver cabin and a lower camera installedin a front vehicle body into one image, determine a position of atransparency processing area in the synthesized image according asteering hinge angle of the front vehicle body, transparency-process atleast one of the first and second images to be transparent in thetransparency processing area according to a position of a bucket or aboom connected to the front vehicle body, and display thetransparency-processed image through a display device.

That is, the position of the transparency processing area in thesynthesized image may be determined to be matched to the steering hingeangle of the front vehicle body, and the at least one of the first imageand the second image may be transparency-processed in the transparencyprocessing area according to the posture of the work apparatus such as aposition of the bucket, to remove a blind spot that is obscured by thefront work apparatus. Thus, an operator's cognitive ability may beincreased to secure stability, to thereby prevent safety accidents.

Further, the transparency processing area may be set according to theoperator's selection, thereby improving the degree of freedom in usingthe transparency processed image, and an efficient system configurationmay be provided.

However, the effect of the inventive concept may not be limited thereto,and may be expanded without being deviated from the concept and thescope of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS

Example embodiments will be more clearly understood from the followingdetailed description taken in conjunction with the accompanyingdrawings.

FIG. 1 is a side view illustrating construction machinery in accordancewith example embodiments.

FIG. 2 is a side view illustrating bucket elevation positions accordingto rotation angles of a boom in FIG. 1 .

FIG. 3 is a plan view illustrating horizontal viewing angles of an uppercamera and a lower camera when the construction machinery of FIG. 1travels straight ahead and turns left.

FIG. 4 is a block diagram illustrating a control system for theconstruction machinery in FIG. 1 .

FIG. 5 is a block diagram illustrating an image processing device inFIG. 4 .

FIG. 6 is a view illustrating a first image captured by the uppercamera.

FIG. 7 is a view illustrating a second image captured by the lowercamera.

FIG. 8 is a view illustrating an image in which the first image of FIG.6 and the second image of FIG. 7 are synthesized by the image processingdevice of FIG. 5 .

FIG. 9 is a flow chart illustrating a control method for a wheel loaderin accordance with example embodiments.

FIG. 10 is a view illustrating a screen on which a first image capturedby an upper camera is displayed on a display device in a cabin when theconstruction machinery travels straight ahead (state A) in FIG. 3 .

FIG. 11 is a view illustrating a screen on which the first imagecaptured by the upper camera is displayed on the display device in thecabin when the construction machinery is steered to the left (state B)in FIG. 3 .

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

Hereinafter, preferable embodiments of the present invention will beexplained in detail with reference to the accompanying drawings.

In the drawings, the sizes and relative sizes of components or elementsmay be exaggerated for clarity.

It will be understood that, although the terms first, second, third,etc. may be used herein to describe various elements, components,regions, layers and/or sections, these elements, components, regions,layers and/or sections should not be limited by these terms. These termsare only used to distinguish one element, component, region, layer orsection from another element, component, region, layer or section. Thus,a first element, component, region, layer or section discussed belowcould be termed a second element, component, region, layer or sectionwithout departing from the teachings of example embodiments.

The terminology used herein is for the purpose of describing particularexample embodiments only and is not intended to be limiting of exampleembodiments. As used herein, the singular forms “a,” “an” and “the” areintended to include the plural forms as well, unless the context clearlyindicates otherwise. It will be further understood that the terms“comprises” and/or “comprising,” when used in this specification,specify the presence of stated features, integers, steps, operations,elements, and/or components, but do not preclude the presence oraddition of one or more other features, integers, steps, operations,elements, components, and/or groups thereof.

Unless otherwise defined, all terms (including technical and scientificterms) used herein have the same meaning as commonly understood by oneof ordinary skill in the art to which example embodiments belong. Itwill be further understood that terms, such as those defined in commonlyused dictionaries, should be interpreted as having a meaning that isconsistent with their meaning in the context of the relevant art andwill not be interpreted in an idealized or overly formal sense unlessexpressly so defined herein.

Example embodiments may, however, be embodied in many different formsand should not be construed as limited to example embodiments set forthherein. Rather, these example embodiments are provided so that thisdisclosure will be thorough and complete, and will fully convey thescope of example embodiments to those skilled in the art.

FIG. 1 is a side view illustrating construction machinery in accordancewith example embodiments. Although a wheel loader 10 is illustrated inFIG. 1 , a control device of construction machinery according to exampleembodiments is not limited to being used only in the wheel loader, butmay be applied to an industrial vehicle such as an excavator, aforklift, etc. Hereinafter, for convenience of description, only thewheel loader 10 will be described.

Referring to FIG. 1 , construction machinery 10 may include a vehiclebody 12, 14, a driver cabin 40, and a front work apparatus. The vehiclebody of the wheel loader 10 in FIG. 1 may include, for example, a frontvehicle body 12 and a rear vehicle body 14 rotatably connected to eachother. The front vehicle body 12 may include the front work apparatusand a front wheel 70. The rear vehicle body 14 may include the drivercabin 40, an engine bay 50 and a rear wheel 72.

The front work apparatus may include a boom 20 and a bucket 30. The boom20 may be freely pivotally attached to the front vehicle body 12, andthe bucket 30 may be freely pivotally attached to an end portion of theboom 20. The boom 20 may be coupled to the front vehicle body 12 by apair of boom cylinders 22, and the boom 20 may be pivoted upwardly anddownwardly by expansion and contraction of the boom cylinders 22. A tiltarm 34 may be freely rotatably supported on the boom 20, almost at itscentral portion. One end portion of the tilt arm 34 may be coupled tothe front vehicle body 12 by a pair of bucket cylinders 32 and anotherend portion of the tilt arm 34 may be coupled to the bucket 30 by a tiltrod, so that the bucket 30 may pivot (crowd and dump) as the bucketcylinders 32 expand and contract.

Additionally, the front vehicle body 12 and the rear vehicle body 14 maybe rotatably connected to each other through a center pin 16 so that thefront vehicle body 12 may swing side to side with respect to the rearvehicle body 14 by expansion and contraction of a steering cylinder (notillustrated).

A travel apparatus for propelling the wheel loader 10 may be mounted atthe rear vehicle body 14. An engine (not illustrated) may be provided inthe engine bay to supply an output power to the travel apparatus. Thetravel apparatus may include a torque converter, a transmission, apropeller shaft, axles, etc. The output power of the engine may betransmitted to the front wheel 70 and the rear wheel 72 through thetorque converter, the transmission, the propeller shaft and the axles,and thus the wheel loader 10 may travels.

A hydraulic pump (not illustrated) for supplying a pressurized hydraulicoil to the boom cylinder 22 and the bucket cylinder 32 of the workapparatus may be mounted at the rear vehicle body 14. The hydraulic pumpmay be driven using at least a portion of the power outputted from theengine. For example, the output power of the engine may drive thehydraulic pump for the work apparatus and a hydraulic pump for thesteering cylinder via a power transmission device such as a gear train.

The hydraulic pump may supply the hydraulic oil to drive the workingdevice, and may be divided into a variable capacity type and a constantcapacity type. A pump control device (EPOS, Electronic Power OptimizingSystem) may be connected to the variable capacity hydraulic pump, and anamount of the hydraulic oil discharged from the variable capacityhydraulic pump may be controlled by the pump control device. A maincontrol valve (MCV) including a boom control valve and a bucket controlvalve may be installed on a hydraulic circuit connected to the hydraulicpump. The hydraulic oil discharged from the hydraulic pump may besupplied to the boom cylinder 22 and the bucket cylinder 32 through theboom control valve and the bucket control valve of the main controlvalve MCV. The main control valve (MCV) may supply the hydraulic oildischarged from the hydraulic pump to the boom cylinder 22 and thebucket cylinder 32 according to a pilot pressure signal in proportion toan operation rate of an operating lever. Thus, the boom 20 and thebucket 30 may be driven by the pressure of the hydraulic oil dischargedfrom the hydraulic pump.

The driver cabin 40 may be installed on the vehicle body of theconstruction machinery, and in case of the wheel loader, the drive cabin40 may be installed on the rear vehicle body 14. A maneuvering devicemay be provided within the driver cabin 40. The maneuvering device mayinclude an acceleration pedal, a brake pedal, an FNR travel lever, theoperating levers for operating cylinders such as the boom cylinder 22and the bucket cylinder 32, etc.

As mentioned above, the wheel loader 10 may include a travelingoperating system for driving the travel apparatus via the powertransmission device and a hydraulic operating system for driving thework apparatus such as the boom 20 and the bucket 30 using the outputpower of the engine 100.

Hereinafter, a control system for the construction machinery will beexplained using the wheel loader as an example.

FIG. 2 is a side view illustrating bucket elevation positions accordingto rotation angles of a boom in FIG. 1 . FIG. 3 is a plan viewillustrating horizontal viewing angles of an upper camera and a lowercamera when the construction machinery of FIG. 1 travels straight aheadand turns left. FIG. 4 is a block diagram illustrating a control systemfor the construction machinery in FIG. 1 . FIG. 5 is a block diagramillustrating an image processing device in FIG. 4 . FIG. 6 is a viewillustrating a first image captured by the upper camera. FIG. 7 is aview illustrating a second image captured by the lower camera. FIG. 8 isa view illustrating an image in which the first image of FIG. 6 and thesecond image of FIG. 7 are synthesized by the image processing device ofFIG. 5 .

Referring to FIGS. 1 to 8 , a control system for the wheel loader mayinclude a camera portion 100 installed in the wheel loader 10 tophotograph the front of the wheel loader 10, an angle informationdetection portion 150 configured to obtain information on refractionangle of the front vehicle body 12, an image processing device 200configured to process an image from the camera portion 100 in real time,and a display device 300 configured to display the image processed bythe image processing device 200. Additionally, the control system forthe wheel loader may further include a work apparatus posture detectionportion configured to detect a posture of the front work apparatusconnected to the front vehicle body 12 and an input portion 400configured to set an image processing condition in the image processingdevice 200.

The image processing device 200 for the wheel loader 10 such as aportion of an engine control unit ECU or a vehicle control unit VCU, ora separate control unit may be mounted in the rear vehicle body 14. Theimage processing device 200 may be implemented with dedicated hardware,software, and circuitry configured to perform the functions describedherein. These elements may be physically implemented by electroniccircuits such as logic circuits, discrete components, microprocessors,hard-wired circuits, memory elements, wiring connections, and the like.

In example embodiments, the camera portion 100 may monitor the front ofthe wheel loader 10 when the wheel loader 10 travels or works, and mayinclude a plurality of cameras. In particular, the camera portion 100may include an upper camera 110 installed in the driver cabin 40 andconfigured to photograph the front of the driver cabin 40 to capture afirst image IM1 and a lower camera 120 installed in the front vehiclebody 12 and configured to photograph the front of the front vehicle body12 to capture a second image IM2. For example, the upper camera and thelower camera may be a fisheye camera having a fisheye lens. Although oneupper camera and one lower camera are illustrated in FIGS. 1 and 2 , itmay not be limited thereto, and a plurality of the upper cameras and aplurality of the lower cameras may be provided.

The upper camera 110 may have a first vertical viewing angle (Field ofView, FoV) θv1 and a first horizontal viewing angle θh1 based on thefront direction of the wheel loader. For example, the first verticalviewing angle and the first horizontal viewing angle may have an angularrange of 60 degrees to 120 degrees. The lower camera 120 may have asecond vertical viewing angle θv2 and a second horizontal viewing angleθh2 based on the front direction of the wheel loader. For example, thesecond vertical viewing angle and the second horizontal viewing anglemay have an angular range of 60 degrees to 120 degrees.

The first image may be an image captured with a focus on a front upperregion through the upper camera 110, and the second image may be animage captured with a focus on a front lower region through the secondcamera 120.

By setting the first vertical viewing angle θv1 of the upper camera 110and the second vertical viewing angle θv2 of the lower camera 120 topartially overlap, the first image and the second image may partiallyoverlap each other.

In example embodiments, the upper camera 110 may be installed tocoincide with or to be at the rear of a central axis (steering centralaxis) of the center pin 16, and the lower camera 120 may be installed tobe in the front of the central axis of the center pin 16.

Since the upper camera 110 and the lower camera 120 are installed atdifferent positions with respect to the steering center axis, asillustrated in FIG. 3 , when the left (or right) refraction of the frontvehicle body 12 is performed, the direction at which the upper camera110 looks and the direction at which the lower camera 120 looks may bedifferent from each other. As will be described later, the imageprocessing device 200 may synthesize the first image IM1 and the secondimage IM2 into one image, and at least one of the first image IM1 andthe second image IM2 in the synthesized image may be processed such thata position of a transparency processing area in which the at least oneimage is transparently processed is adjusted to be matched to a steeringhinge angle (θs) of the wheel loader 10.

In example embodiments, the work apparatus posture detection portion maydetect whether the front work apparatus invades the transparencyprocessing area in a display area of the display device 300. Asdescribed later, transparency processing may be performed on thecaptured image when the work apparatus invades a predetermined position(position matched to the steering hinge angle), that is, an actualposition corresponding to the predetermined transparency processing areaamong the entire display area of the display device 300, so that anoperator's view may be secured. The posture of the front work apparatusmay include a position of the bucket 30 (a height of the bucket from theground) or a posture of the boom 20 (a rotation angle of the boom). Tothis end, the work apparatus posture detection portion may include aboom angle sensor 24 for detecting the position of the bucket 30 or theposture of the boom 20. In addition, the work apparatus posturedetection portion may include a bucket angle sensor (not illustrated)for detecting a relative rotation angle between the boom 20 and thebucket 30. The work apparatus posture detection portion may include adisplacement sensor for detecting a stroke of the cylinder driving theboom 20, in place of the boom angle sensor 24.

Further, the work apparatus posture detection portion may include animage analysis device (for example, shape recognition portion) thatanalyzes an image of the front work apparatus captured through thecamera to determine the posture of the front work apparatus.

The boom angle sensor 24 may detect the rotation angle of the boom 20and provide information on the position of the bucket 30 based on therotation angle of the boom 20. As illustrated in FIG. 2 , the rotationangle of the boom 20 may be an angle θ between an extension line L atthe lowest position (0%) of the boom 20 (bucket 30) and an extensionline R at an elevated position of the boom 20. The rotation angle of theboom 20 at the highest position of the boom 20 (max boom height) isθmax.height, and in this case, the boom (bucket) position may be themaximum height (100%).

In example embodiments, the angle information detection portion 150 mayinclude a sensor for detecting refraction angle information of the frontvehicle body 12 with respect to the rear vehicle body 14 by a steeringdevice. The detection value from the sensor may be transmitted to theimage processing device 200 through a CAN network.

For example, the angle information detection portion 150 may include acenter pin angle sensor, a steering cylinder displacement sensor, or agyro sensor. The center pin angle sensor may be installed in the centerpin 16 connecting the front vehicle body 12 and the rear vehicle body 14to detect a steering hinge angle of the front vehicle body 12. Thesteering cylinder displacement sensor may include a stroke sensor fordetecting a stroke of a cylinder inside the steering cylinder. The gyrosensor may include a first sensor and a second sensor installed in thefront vehicle body 12 and the rear vehicle body 14 respectively, torecognize absolute angles of 6 or more axes, and may detect relativeangles between the first and second sensors to recognize the steeringhinge angle.

In example embodiments, the image processing device 200 may synthesizethe first image IM1 and the second image IM2 captured by the uppercamera 110 and the lower camera 120 into one image, may determine thesteering hinge angle of the front vehicle body 12 from the angleinformation of the angle information detection portion 150, and maydetermine the position of the transparency processing area in which atleast one of the first and second images in the synthesized image istransparency-processed according to the steering hinge angle. The imageprocessing device 200 may include a steering angle calculator 210, animage synthesizer 220, a transparency processor 230, an image renderingportion 240 and a storage portion 250. The image processing device 200may be installed in the form of a control device embedded in the controldevice or the display device of the construction machinery.

In particular, the steering angle calculator 210 may determine thesteering hinge angle θs of the front vehicle body 12 from the refractionangle information detected by the angle information detection portion150. The steering angle calculator 210 may convert the detection valuefrom the angle information detection portion 150 into the steering hingeangle using a conversion table stored in the storage portion 250.

The image synthesizer 220 may synthesize the first image IM1 and thesecond mage IM2 into one image. The image synthesizer 220 may match thefirst image and the second image captured by the upper camera 110 andthe lower camera 120 to find portions of images that overlap (areduplicated) in the first and second images and match the overlappingportions of the images into one synthesized image. The transparencyprocessor 220 may transparency-process at least one of the first andsecond images in the transparency processing area. The image renderingportion 230 may render the image-processed synthetic image into a 3Dimage. The image rendering portion 230 may process the synthesized imageto be displayed like a real image and output the rendering-processedsynthesized image to the display device 300. The functions of the imagesynthesizer 220, the transparency processor 230 and the image renderingportion 240 may be implemented through a single processor such as GP orCPU for image processing, or through computational processing ofseparate processors.

In example embodiments, the transparency processor 230 may performtransparency processing in response to a case in which at least aportion of the front work apparatus invades a position corresponding tothe transparency processing area. When the bucket or boom position islower than a predetermined position (transparency switching position),which can be determined that the at least a portion of the front workapparatus does not invade the transparency processing area, the secondimage in the synthesized image may be transparency-processed to betransparent. On the other hand, when the bucket or boom position ishigher than the predetermined position (transparency switchingposition), which can be determined that the at least a portion of thefront work apparatus invades the transparency processing area, the firstimage in the synthesized image may be transparency-processed to betransparent. For example, the predetermined position of the boom may beset such that the rotation angle θ of the boom 20 is within a range of15 degrees to 20 degrees.

When the bucket 30 is positioned between the lowest position (0%) andthe predetermined bucket position, that is, the transparency switchingposition which is the boundary of the transparency processing area, thesecond image captured from the lower camera 120 may betransparency-processed, so that an object implemented by the uppercamera 110 may be displayed as a main point (focus). In the second imagecaptured from the lower camera 120, when the bucket 30 is in arelatively low position, the front view of the front vehicle body 12 maybe obscured by the front work apparatus including the boom 20 and thebucket 30. The transparency processor 220 may process the second imageto be transparent and display the first image as a focus to therebyprevent the front view from being obscured by the front work apparatus.

When the bucket 30 is positioned between the predetermined bucketposition and the highest position (100%) of the transparency processingarea, the first image captured from the upper camera 110 may betransparency-processed, so that an object implemented by the lowercamera 120 may be displayed as a main point (focus). In the first imagecaptured from the upper camera 110, when the bucket 30 is in arelatively high position, the front view of the front vehicle body 12may be obscured by the front work apparatus including the boom 20 andthe bucket 30. The transparency processor 220 may process the firstimage to be transparent and display the second image as a focus tothereby prevent the front view from being obscured by the front workapparatus.

When the bucket 30 is lifted or lowered to pass through thepredetermined bucket position (transparency switching position), animage located in the transparency processing area transparency-processedby the transparency processor 220 may be converted from the second imageto the first image or from the first image to the second image.

Alternatively, the transparency processor 230 may transparency-processthe second image in the synthesized image to be transparent when therotation angle θ of the boom is within a first angle range,transparency-process the first and second images in the transparencyprocessing area of the synthesized image to be transparent when therotation angle θ of the boom is within a second angle range, andtransparency-process the first image in the synthesized image to betransparent when the rotation angle θ of the boom is within a thirdangle range. For example, the first angle range may be within 0 degreeto 15 degrees, the second angle range may be within 15 degrees to 25degrees, and the third angle range may be within 25 degrees to 45degrees.

In example embodiments, an image processing condition in the imageprocessing device 200 may be set through the input portion 400. Forexample, the image processing condition may include a location, a size,etc. of the transparency processing area. As the transparency processingarea is determined, the transparency switching position of the first andsecond images, the transparency processing area in the entire displayarea of the display device 300, and the like may be set. For example,the transparency switching position may represent a boundary position ofthe transparency processing area, and when the bucket 30 moves to belocated at the boundary of the transparency processing area, the bucket30 may be considered to be located at a predetermined position fortransparency switching. The size and location of the transparencyprocessing area, the transparency switching timing, etc. may be fixedlyset by a manufacturer according to a type of equipment, and may befreely changed and set by the operator or maintenance personnel.

For example, the input portion 400 may be implemented in a form of aninstrument panel option, and the operator may change the timing pointfor the transparency switching, the area to be processed fortransparency, and the like through the input portion 400.

As mentioned above, when the transparency processing area and thetransparency switching timing are set, the display device 300 maydisplay an image by dividing the image captured by the camera portioninto the transparency processing area R and an external area of thetransparency processing area R. The display device 300 may additionallydisplay an outline of the transparency processing area R such that thetransparency processing area R can be distinguished, or may not displaythe outline of the transparency processing area and may display thetransparency-processed image to be connected to an image of the externalarea of the transparency processing area R.

Additionally, the display device 300 may display the first image in theexternal area of the transparency processing area R, and may display atransparency image in which at least one of the first image and thesecond image is displayed as a focus according to the progress of thetransparency processing, within the transparency processing area R.

For example, when the bucket 30 is located in the external area of thetransparency processing area R, the display device 300 may display onlythe first image that interconnects the transparency processing area Rand the external area of the transparency processing area R.Alternatively, a transparency-processed image in which the first imageis displayed as a focus may be displayed within the transparencyprocessing area R. In this case, the operator may recognize that thedisplay device 300 displays the first image as a whole due to thetransparency image in which the first image is displayed as the focus.Additionally, when at least a portion of the bucket 30 is located withinthe transparency processing area R, the display device 300 may display atransparency-processed image in which the second image is displayed as afocus or the second image within the transparency processing area R, andmay display the first image in which only the image in the transparencyprocessing area R is excluded, in the external area of the transparencyprocessing area.

In example embodiments, the transparency processor 230 may determine theposition of the transparency processing area in the synthesized imageaccording to the refraction angle information of the front vehicle bodyobtained from the angle information detection portion 150. Thetransparency processor 230 may adjust the position of the transparencyprocessing area according to the steering hinge angle θs of the frontvehicle body calculated by the steering angle calculator 210.

The position of the transparency processing area may be adjusted tofollow the calculated steering hinge angle. As the stroke of thesteering cylinder is changed by the steering device, the front vehiclebody 12 may be bent to the left (or right), and at this time, the frontworking apparatus including the boom 20 and the bucket 30 may deviatefrom the initially set transparency processing area to obscure the frontview. In case of the steering cylinder displacement sensor, the steeringangle calculator 210 may convert the stroke value of the steeringcylinder into the steering hinge angle. The transparency processor 230may automatically change the position of the transparency processingarea to follow the converted steering hinge angle.

As illustrated in FIGS. 6 to 8 , when a portion of the boom or thebucket is positioned in the transparency processing area R of the firstimage M1, a portion of the second image may be cropped based on thefirst image, and then, the cropped image of the second image may besynthesized with the first image.

First, the position of the transparency processing area R may be setaccording to the detected posture (position of the boom or bucket) ofthe work apparatus and the refraction angle information of the frontvehicle body. In FIGS. 6 to 8 , for convenience of explanation, thefirst image captured by the upper camera, the second image captured bythe lower camera and the synthesized image when the constructionmachinery is in the straight traveling state (state A) in FIG. 3 areillustrated. In this case, the transparency processing area R may belocated at an initial position, for example, in the center of the screenof the display device 300.

Then, as illustrated in FIG. 7 , a partial region in the second imagecorresponding to the transparency processing area may be determined as acrop region CR, and a crop image may be extracted based on thedetermined crop region CR. Then, the transparency of a region in thefirst image of FIG. 6 corresponding to the transparency processing areaR may be adjusted. The transparency of the region the first imagecorresponding to the transparency processing area R may be adjusted to,for example, 180/255 level. In this case, the transparency of thecropped image extracted from the second image may not be adjusted.

Then, as illustrated in FIG. 8 , the cropped image extracted from thesecond image may be synthesized in the transparent processing region Rof the first image. In this case, when the area of the crop region CR issmaller than the area of the transparent processing region R in thefirst image, the crop image may be enlarged and the enlarged crop imagemay be synthesized with the first image.

If there is a person O in front of the wheel loader, in the first imageof FIG. 6 , the person O cannot be accurately identified because theperson is covered by the boom or bucket, but in the second image of FIG.7 , the person O can be identified. The cropped image extracted from thesecond image of FIG. 7 may include an image of the person O. Since thecropped image is synthesized into the region corresponding to thetransparency processing area of the first image, the image of the personO extracted from FIG. 7 can be identified in the transparency processingarea of the synthesized image of FIG. 8 . Accordingly, the forward fieldof view limited by the working apparatus may be improved.

In FIG. 3 , when the construction machinery is in the left or rightsteering state, the transparency processing area R may move to the leftor right side in the screen of the display device 300 according to therefraction angle information of the front vehicle body. In this case,the position of the transparency processing area R may be moved to theleft or right according to the refraction angle information, and atleast one of the first image and the second image may be processed to betransparent in the moved transparency processing area R.

In example embodiments, the image processing device 200 may recognize aperson or other object (equipment, vehicle, etc.) from the first image,the second image and the synthesized image, and may process that therecognized object is displayed with a mark such as a bounding box (D inFIG. 8 ) or an outline of the recognized object is displayed with adotted line or the like.

The image processing device 200 may compare an actual image in the imagewith a learning image of the object stored in the storage portion 250,and when it is determined that the actual image and the learning imageof the object are the same as each other, the object may be recognized.Here, the learning image of the object may include images stored bymachine learning various shapes photographed by a camera. Machinelearning may be a field of artificial intelligence and may refer to analgorithm that enables a processing device such as a computer to learn.

Hereinafter, a method of controlling construction machinery using thecontrol system for construction machinery in FIG. 4 will be explained.The following description will also be described based on the wheelloader as in the above-described method.

FIG. 9 is a flow chart illustrating a control method for a wheel loaderin accordance with example embodiments. FIG. 10 is a view illustrating ascreen on which a first image captured by an upper camera is displayedon a display device in a cab when the construction machinery travelsstraight ahead (state A) in FIG. 3 . FIG. 11 is a view illustrating ascreen on which the first image captured by the upper camera isdisplayed on the display device in the cabin when the constructionmachinery is steered to the left (state B) in FIG. 3 .

Referring to FIGS. 1 to 11 , first, a first image IM1 and a second imageIM2 may be obtained respectively through an upper camera 110 and a lowercamera 120 installed in a wheel loader 10 (S100), and a refraction angleinformation of a front vehicle body 12 with respect to a rear vehiclebody 14 may be obtained (S110). The first image IM1 and the second imageIM2 may be synthesized into one image (S120), and a position of atransparency processing area may be determined according to therefraction angle information (S130).

In example embodiments, the first image IM1 for the front of a drivercabin 40 may be obtained using the first camera 110 installed in thedriver cabin 40. The second image IM2 for the front of the front vehiclebody 12 may be obtained using the second camera 120 installed in thefront vehicle body 12.

The first image may be an image captured with a focus on a front upperregion of the wheel loader through the upper camera 110, and the secondimage may be an image captured with a focus on a front lower regionthrough the second camera 120. A first vertical viewing angle θv1 of theupper camera 110 and a second vertical viewing angle θv2 of the lowercamera 120 may be set to partially overlap and a first horizontalviewing angle θh1 of the upper camera 110 and a second horizontalviewing angle θh2 of the lower camera 120 may be set to partiallyoverlap, the first image and the second image may partially overlap eachother.

For example, the upper camera 110 may be installed to coincide with orto be at the rear of a central axis (steering central axis) of thecenter pin 16, and the lower camera 120 may be installed to be in thefront of the central axis of the center pin 16. Since the upper camera110 and the lower camera 120 are installed at different positions withrespect to the steering center axis, as illustrated in FIG. 3 , when theleft (or right) refraction of the front vehicle body 12 is performed,the direction at which the upper camera 110 looks and the direction atwhich the lower camera 120 looks may be different from each other.

In example embodiments, the refraction angle information of the frontvehicle body 12 with respect to the rear vehicle body 14 may be obtainedfrom an angle information detection portion 150. The angle informationdetection portion 150 may include a sensor for detecting the refractionangle information of the front vehicle body 12 with respect to the rearvehicle body 14 by a steering device. The detection value from thesensor may be transmitted to an image processing device 200 through aCAN network.

In example embodiments, the image processing device 200 may match thefirst image IM1 and the second image IM2 to synthesize the first imageand the second mage into one image. Additionally, the image processingdevice 200 may a steering hinge angle θs of the front vehicle body 12from the refraction angle information obtained by the angle informationdetection portion 150.

A steering angle calculator 210 of the image processing device 200 maydetermine the steering hinge angle θs of the front vehicle body 12 fromthe refraction angle information detected by the angle informationdetection portion 150. The steering angle calculator 210 may convert thedetection value from the angle information detection portion 150 intothe steering hinge angle using a conversion table stored in a storageportion 250.

A transparency processor 230 of the image processing device 200 maydetermine a position of the transparency processing area in thesynthesized image according to the refraction angle information of thefront vehicle body obtained from the angle information detection portion150. The transparency processor 230 may adjust the position of thetransparency processing area according to the steering hinge angle θs ofthe front vehicle body calculated by the steering angle calculator 210.

The image processing apparatus 200 may set a portion of the first imageIM1 from the upper camera 110 as a first synthesis region, may set aportion of the second image IM2 from the lower camera 120 as a secondsynthesis region, may translucently process the second synthesis regionof the second image IM2, and may the translucently processed secondsynthesis region may be synthesized with the first synthesis region ofthe first image IM1.

As illustrated in FIG. 10 , when the wheel loader 10 travels straight,the transparency processor 230 of the image processing device 200 maymaintain the position of the transparency area R as an initial positionto follow the calculated steering hinge angle.

Additionally, in the transparency processing area, a portion in any oneselected from the first image and the second image captured by the uppercamera 110 and the lower camera 120 or portions of the first and secondimages may be processed to be transparent. For example, a portion of theimage including the boom or bucket of the front working apparatus may beselectively processed to be transparent in the transparency processingarea.

As illustrated in FIG. 11 , when the front vehicle body 12 is refractedto the left by the steering device, the front working apparatusincluding the boom 20 and the bucket 30 may deviate from the initiallyset transparency processing area, so that the front view is obscured. Atthis time, the steering angle calculator 210 of the image processingdevice 200 may calculate the steering hinge angle θs from the detectionvalue from the angle information detection portion 150, and thetransparency processor 230 of the image processing device 200 mayautomatically change the position of the transparency processing area Rto follow the calculated steering hinge angle.

Then, at least one of the first and second images may be transparencyprocessed in the transparency processing are R (S140), and thetransparency-processed synthesized image may be displayed through thedisplay device 300 (S150).

In example embodiments, a posture of the front working apparatus may bedetected. A rotation angle of a boom 20 connected to the front vehiclebody 12 may be detected. Information on a position of a bucket 30, thatis, a height of the bucket 30 from the ground may be detected by a boomangle sensor 24. An elevated height of the bucket may be determined fromthe rotation angle of the boom 20 detected by the boom angle sensor 24.

In example embodiments, an image portion of the boom or bucket that is aportion of the front working apparatus among the first and second imagesin the transparency processing area R may be selectively processed to betransparent (S140), and the transparency-processed synthesized image maybe displayed through the display device 300 (S150).

In the transparency processing area R, an outline of an exterior of theboom or bucket of the transparency-processed first and second images maybe displayed with a line, a dotted line, or a shade, so that only theshape can be identified.

The outline of the boom or bucket displayed in the transparentprocessing area R may be taken from the image taken from the actualfirst image or the second image, and may be matched with the movement ofthe boom or bucket according to the actual operation of a driver.

In example embodiments, the image processing device apparatus 200 mayrecognize an object in the first image or the second image which are tobe transparency-processed, and may process that an outline of therecognized object is displayed with a dotted line or the like, or therecognized object is displayed with a notation such as a bounding box.For example, a computing device may identify an object in an image basedon a predetermined algorithm or the like, and may generate a synthesizedimage by displaying an outline of the object. In this case, the drivermay simply check a partial shape of the transparency-processed wheelloader 200 by using a dotted outline or the like.

As illustrated in FIG. 2 , the rotation angle of the boom 20 may be anangle θ between an extension line L at the lowest position (0%) of theboom 20 and an extension line R at an elevated position of the boom 20.The rotation angle of the boom 20 at the highest position of the boom 20(max boom height) is θmax.height, and in this case, the bucket positionmay be the maximum height (100%).

Then, whether or not the bucket position is lower than a predeterminedposition (transparency switching position) may be determined. Thepredetermined position may be the transparency switching position whichis the boundary of the transparency processing area. That is, thecomparison between the position of the bucket and the predeterminedposition may include checking whether a portion of the bucket 30 or theboom 20 is located within the transparency processing area R. When thebucket or the boom is lower than the predetermined position, the secondimage in the synthesized image may be transparency processed, and whenthe bucket or the boom is higher than the predetermined position, thefirst image in the synthesized image may be transparency processed.Here, the predetermined position may be a lower boundary of thepredetermined transparency processing area R based on an image displayedthrough the display device 300. Then, the transparency-processedsynthesized image may be displayed through the display device 300. Inthis case, the display device 300 may display the first image in anexternal area of the transparency processing area R.

In example embodiments, the image processing device 200 may performtransparency processing at least one of the first and second images tobe transparent in the synthesized image according to the detected boomposition.

The transparency processor 230 may transparency-process the first andsecond images to be transparent only in the transparency processing areaR, that is, a partial area of the entire display area of the displaydevice 300. The transparency processing area R may be defined to includean area in which the front view is obscured by the front work apparatusincluding the elevating boom 20 and the bucket 30.

In the transparency processing, the portions of the first image and/orthe second image within the transparency processing area R of thesynthesized image may be removed or translucently processed to overlapthe background image, or an outline of an exterior of the first imageand/or the second image may be two-dimensionally drawn with a line ordotted line so that only the shape may be identified. For example, theportions of the first image or the second image in the transparencyprocessing area may be removed from the synthesized image using an alphablending technique.

When the bucket 30 or the boom 20 is positioned between the lowestposition (0%) and the predetermined bucket or boom position, the secondimage captured from the lower camera 120 may be transparency-processed,so that an object implemented by the upper camera 110 may be displayedas a main point (focus) within the transparency processing area R of thedisplay device 300. When the bucket 30 or the boom 20 is in a relativelylow position, a portion of the front work apparatus obscuring the frontview in the second image may be transparency-processed so that theobject may be identified in the synthesized image.

When the bucket 30 or the boom 20 is positioned between thepredetermined position and the highest position (100%), the first imagecaptured from the upper camera 110 may be transparency-processed, sothat an object implemented by the lower camera 120 may be displayed as amain point (focus) within the transparency processing area R of thedisplay device 300. When the bucket 30 or the boom 20 is in a relativelyhigh position, a portion of the front work apparatus obscuring the frontview in the first image may be transparency-processed so that the objectmay be identified in the synthesized image.

For example, the predetermined position of the boom may be set such thatthe rotation angle θ of the boom 20 is within a range of 15 degrees to20 degrees.

Alternatively, the second image in the synthesized image may betransparency-processed to be transparent when the rotation angle θ ofthe boom is within a first angle range, the first and second images inthe transparency processing area of the synthesized image may betransparency-processed to be transparent when the rotation angle θ ofthe boom is within a second angle range, and the first image in thesynthesized image may be transparency-processed to be transparent whenthe rotation angle θ of the boom is within a third angle range. Forexample, the first angle range may be within 0 degree to 15 degrees, thesecond angle range may be within 15 degrees to 25 degrees, and the thirdangle range may be within 25 degrees to 45 degrees.

In example embodiments, an image processing condition for transparencyprocessing the first and second images may be set. The image processingcondition in the image processing device 200 may be set through an inputportion 400. For example, the image processing condition may include alocation, a size, etc. of the transparency processing area. Atransparency switching timing of the first and second images may bedetermined based on the position of the bucket 30 or the bucket 20 andthe predetermined bucket or boom position. The transparency processingarea may be selected according to a type of equipment.

For example, the input unit 400 may be implemented in a form of aninstrument panel option, and the operator may change the timing pointfor the transparency switching, the area to be processed fortransparency, and the like through the input portion 400. The inputportion 400 may be provided in a form of a separate manipulation deviceprovided in the driver cabin, a manipulation device integrally providedwith the display device, or a touch screen constituting a display screenof the display device. Thus, the operator may set various imageprocessing conditions such as setting a periphery of an object requiringattention during work as the transparent processing area.

As mentioned above, the first image and the second image captured fromthe upper camera 110 installed in the driver cabin 40 of the wheelloader 10 and the lower camera 120 installed in the front vehicle body12 may be synthesized into one image, the position of the transparentprocessing region R in the synthesized image may be determined accordingto the steering hinge angle θs of the front vehicle body 12, at leastone of the first and second images may be transparency-processed to betransparent in the synthesized image according to the position of thebucket 30 or the boom 20 connected to the front vehicle body 12, and thetransparency-processed image may be displayed through the display device300.

When the bucket 30 or the boom 20 is in a relatively low positionbetween the lowest position (0%) and the predetermined bucket position,in the second image captured from the lower camera 120, the front viewof the front vehicle body 12 may be obscured by the front work apparatusincluding the boom 20 and the bucket 30. When the bucket 30 is in arelatively high position between the predetermined bucket position andthe highest position (100%) of the transparency processing area, in thefirst image captured from the upper camera 110, the front view of thefront vehicle body 12 may be obscured by the front work apparatusincluding the boom 20 and the bucket 30.

The first image and/or the second image may be transparency-processed inthe synthesized image according to the position of the bucket 30 or theboom 20, to remove a blind spot that is obscured by the front workapparatus.

Further, since the mounting positions of the upper camera 110 and thelower camera 120 are different from each other, when the constructionmachinery is steered to the left or right, the bucket 30 may deviatefrom the initially set the transparent processing area R. When the wheelloader 10 is steered, the steering hinge angle θs of the front vehiclebody 12 may be determined according to the refraction angle informationof the front vehicle body 12 with respect to the rear vehicle body 14,and in the image in which the first image IM1 and the second image IM2are synthesized, the image may be processed so that the position of thetransparent processing region R is changed to be matched to the steeringhinge angle θs.

Accordingly, it may be possible to prevent the front view from beingobscured by the front working apparatus including the boom 20 and thebucket 30 event when the wheel loader 10 is steered. Thus, theoperator's cognitive ability may be increased to secure stability, tothereby prevent safety accidents.

The foregoing is illustrative of example embodiments and is not to beconstrued as limiting thereof. Although a few example embodiments havebeen described, those skilled in the art will readily appreciate thatmany modifications are possible in example embodiments withoutmaterially departing from the novel teachings and advantages of thepresent invention. Accordingly, all such modifications are intended tobe included within the scope of example embodiments as defined in theclaims.

What is claimed is:
 1. A control system for construction machinery, thecontrol system comprising: an upper camera installed in a driver cabinof a rear vehicle body to photograph the front of the driver cabin; alower camera installed in a front vehicle body rotatably connected tothe rear vehicle body to photograph the front of the front vehicle body;an angle information detection portion configured to detect informationon a refraction angle of the front vehicle body with respect to the rearvehicle body; an image processing device configured to synthesize firstand second images captured from the upper camera and the lower camerainto one image, and configured to determine a position of a transparencyprocessing area in the synthesized image according to the refractionangle information and transparency-process at least one of the first andsecond images in the transparency processing area; and a display deviceconfigured to display the synthesized image transparency-processed bythe image processing device.
 2. The control system of claim 1, whereinthe image processing device includes: a steering angle calculatorconfigured to determine a steering hinge angle of the front vehicle bodyfrom the refraction angle information obtained by the angle informationdetection portion; and a transparency processor configured to determinethe position of the transparency process area in the synthesized imageaccording to the determined steering hinge angle.
 3. The control systemof claim 2, wherein the steering angle calculator converts a detectionvalue obtained from the angle information detection portion into thesteering hinge angle value of the front vehicle body.
 4. The controlsystem of claim 1, wherein the angle information detection portionincludes a center pin angle sensor, a steering cylinder displacementsensor or a gyro sensor.
 5. The control system of claim 1, furthercomprising: a work apparatus posture detection portion configured todetect a posture of a front work apparatus, and wherein the imageprocessing device transparency-processes the at least one of the firstand second images in the synthesized image according to the posture ofthe front work apparatus detected by the work apparatus posturedetection portion.
 6. The control system of claim 5, wherein the imageprocessing device transparency-processes the first image in thesynthesized image when at least a portion of the front work apparatusinvades a predetermined position, and the image processing devicetransparency-processes the second image in the synthesized image whenthe work apparatus does not invade the predetermined position
 7. Thecontrol system of claim 1, further comprising: an input portionconfigured to set an image processing condition in the image processingdevice.
 8. The control system of claim 7, wherein the image processingcondition includes a transparency processing switching timing of thefirst and second images or the transparency processing area of theentire display area of the display device.
 9. The control system ofclaim 1, wherein the image processing device processes that, in thetransparency processing area, an outline of an exterior of a boom or abucket of the transparency-processed first and second images isdisplayed with a line or a dotted line.
 10. The control system of claim9, wherein the outline of the transparency-processed boom or bucket isdisplayed by transparency-processing an image of the boom or bucket thatis taken from the first image or the second image.
 11. The controlsystem of claim 1, wherein the image processing device selectivelytransparency-processes an image of a boom or a bucket coupled to thefront vehicle body in the first and second images in the transparentprocessing area.
 12. The control system of claim 1, wherein thesynthesized image includes an object recognized by the image processingdevice in the first image and the second image.
 13. The control systemof claim 1, wherein the image processing device recognizes a person,animal, building or equipment as an object through a predeterminedalgorithm.
 14. The control system of claim 1, wherein the imageprocessing device extracts a crop image from any one of the first andsecond images, transparency-process a partial region in the other one ofthe first and second images, and places the extracted crop image in thetransparency-processed region to create the synthesized image.
 15. Thecontrol system of claim 14, wherein the transparency-processed regioncorresponds to the transparency processing area.
 16. The control systemof claim 1, wherein the image processing device sets a portion of thefirst image as a first synthesis region, sets a portion of the secondimage as a second synthesis region, translucently processes the secondsynthesis region of the second image, and synthesizes the translucentlyprocessed second synthesis region with the first synthesis region of thefirst image to create the synthesized image.
 17. A method of controllingconstruction machinery, the method comprising: obtaining a first imageof the front of a driver cabin from an upper camera installed in thedrive cabin of a rear vehicle body; obtaining a second image of thefront of a front vehicle body from a lower camera installed in the frontvehicle body rotatably connected to the rear vehicle body; obtaininginformation on a refraction angle of the front vehicle body with respectto the rear vehicle body; synthesizing the first and second images intoone image; determining a position of a transparency processing area inthe synthesized image according to the obtained refraction angleinformation; transparency-processing at least one of the first andsecond images in the transparency processing area; and displaying thetransparency-processed image through a display device.
 18. The method ofclaim 17, wherein determining the position of the transparencyprocessing area according to the obtained refraction angle informationincludes: determining a steering hinge angle of the front vehicle bodyfrom the refraction angle information; and determining the position ofthe transparency process area in the synthesized image according to thedetermined steering hinge angle.
 19. The method of claim 18, whereindetermining the steering hinge angle of the front vehicle body from therefraction angle information includes converting a refraction angledetection value into the steering hinge angle value of the front vehiclebody.
 20. The method of claim 17, wherein obtaining the refraction angleinformation of the front vehicle body with respect to the rear vehiclebody includes using a refraction angle detection sensor, and therefraction angle detection sensor includes a center pin angle sensor, asteering cylinder displacement sensor or a gyro sensor.
 21. The methodof claim 17, further comprising: detecting a posture of a front workapparatus, and wherein transparency-processing the at least one of thefirst and second images in the transparency processing area includestransparency-processing the at least one of the first and second imagesin the synthesized image according to the detected posture of the frontwork apparatus.
 22. The method of claim 17, further comprising: settingan image processing condition under which the at least one of the firstand second images is transparency processed.
 23. The method of claim 22,wherein the image processing condition includes a transparencyprocessing switching timing of the first and second images or thetransparency processing area of the entire display area of the displaydevice.
 24. A method of controlling construction machinery, the methodcomprising: obtaining a first image of the front of a driver cabin froman upper camera installed in the drive cabin of a rear vehicle body;obtaining a second image of the front of a front vehicle body from alower camera installed in the front vehicle body rotatably connected tothe rear vehicle body; obtaining information on a refraction angle ofthe front vehicle body with respect to the rear vehicle body;determining a position of a transparency processing area in the firstimage according to the obtained refraction angle information; extractinga crop image by setting a partial region in the second imagecorresponding to the transparency processing area as a crop region and;adjusting transparency of the transparency processing area of the firstimage; synthesizing the crop image extracted from the second image tothe transparency processing area of the first image; and displaying thesynthesized image through a display device.
 25. The method of claim 24,wherein synthesizing the extracted cropped image to the transparencyprocessing area of the first image includes: adjusting a size of thecrop image extracted from the second image; and synthesizing theadjusted crop image to the transparency processing area of the firstimage.
 26. The method of claim 24, wherein adjusting the transparency ofthe transparency processing area of the first image includes processingsuch that an outline of an exterior of a boom or a bucket is displayedwith a line or a dotted line in the transparency processing area of thefirst image.
 27. The method of claim 24, further comprising: recognizinga person, animal, building or equipment as an object through apredetermined algorithm in the synthesized image.